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Abstract 

We have developed a steady state theory of complex transport net- 
works used to model the flow of commodity, information, viruses, opin- 
ions, or traffic. Our approach is based on the use of the Markov chains 
defined on the graph representations of transport networks allowing for 
the effective network design, network performance evaluation, embedding, 
partitioning, and network fault tolerance analysis. Random walks embed 
graphs into Euclidean space in which distances and angles acquire a clear 
statistical interpretation. Being defined on the dual graph representations 
of transport networks random walks describe the equilibrium configura- 
tions of not random commodity ffows on primary graphs. This theory 
unifies many network concepts into one framework and can also be ele- 
gantly extended to describe networks represented by directed graphs and 
multiple interacting networks. 



PACS codes: 89.75.Fb, 89.75.-k, 89.90.+n 
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1 Introduction 

Transport networks are used to model the flow of commodity, information, 
viruses, opinions, or traffic. They typically represent the networks of roads, 
streets, pipes, aqueducts, power lines, or nearly any structure which permits 



either vehicular movement or flow of some commodity, products, goods or ser- 
vice. The major aim of the analysis is to determine the structure and properties 
of transport networks that are important for the emergence of complex flow 
patterns of vehicles (or people) through the network such as the Braess paradox 
[T]. This counter-intuitive phenomenon occurs when adding more resources to 
a transportation network (say, a new road or a bridge) deteriorates the quality 
of traffic by creating worse delays for the drivers, rather than alleviate it. The 
Braess paradox has been observed in the street vehicular traffic of New York 
City and Stuttgart, [5]. 

The paper is partitioned into three sections. In Sec. [21 we demonstrate that 
Markov chains arise naturally in the problem of network equilibriums. Random 
walks embed connected undirected graphs into Euclidean space that can be 
used in order to investigate them. In Sec. [3l we discuss the thermodynamics 
of random walks defined on the undirected graph representations of transport 
networks. In SecIH we extend the approach to networks represented by directed 
graphs and to multiple interacting networks. We conclude in the last section. 

2 Euclidean Space Associated to Transport Net- 
works 

The main goal of complex network theory is to study relationships between 
parts of complex systems that gives rise to the patterns of collective behavior 
and of interactions between complex systems with their environments. 

2.1 Traffic equilibrium, space syntax, and random walks 

Given a connected undirected graph G(V, E), in which V is the set of nodes 
and E the set of edges, we introduce the traffic volume f : E ^ {0,oo[ through 
every edge e G E. It then follows from the Perron-Frobenius theorem that the 
linear equation 

/(e) = E /(e')cxp(-/i^(e')) (1) 

e' eE 

has a unique positive solution /(e) > 0, for every edge e Cz E, for a fixed positive 
constant h > and a chosen set of positive metric length distances i{e) > 0. 
This solution is naturally identified with the traffic equilibrium state of the 
transport network deflned on G, in which the permeability of edges depends 
upon their lengths. The parameter h is called the volume entropy of the graph 
G, while the volume of G is defined as the sum 

Vol(G) = IY. ^(^)- 

The volume entropy h is defined to be the exponential growth of the balls in a 
universal covering tree for G with the lifted metric, [3]- [6]. 
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The degree of a node v gV is the number of its neighbors in G, deg(w) — ky. 
It has been shown in [6] that among all undirected connected graphs of normal- 
ized volume, Vol(G) = 1, which are not cycles and ky 1 for all nodes, the 
minimal possible value of the volume entropy, min(/i) — i X^jjgi^ ^'^S (J^v — 1) 
is attained for the length distances 

) ^ log ((fc,(e) - 1) (fct(e) - 1)) ^ 

2 min(/i) ' 

where i{e) G V and t{e) G V arc the initial and terminal vertices of the edge 
e G E respectively. It is then obvious that substituting ^ and mm{h) into H]) 
the operator exp {—h£{e')) is given by a symmetric Markov transition operator, 

m = E . ^^'^ (3) 

e'eE ^ (fcj(e') - l) (fct(e') - l) 

which rather describes time reversible random walks over edges than over nodes. 
In other words, we are invited to consider random walks on the dual graphs. 
The flows satisfying ([T]) with the operator ([3]) meet the mass conservation prop- 
erty, fij — ■"'j' Sjev'^'i = 1- The Eq.([3]) unveils the indispensable role 
Markov's chains defined on edges play in equilibrium traffic modelling and ex- 
poses the degrees of nodes as a key determinant of the transport networks prop- 
erties. 

The notion of traffic equilibrium had been introduced by J.G. Wardrop in 
and then generalized in 8^ to a fundamental concept of network equilibrium. 
Wardrop's traffic equilibrium is strongly tied to the human apprehension of space 
since it is required that all travellers have enough knowledge of the transport 
network they use. The human perception of places is not an entirely Euclidean 
one, but are rather related to the perceiving of the vista spaces (streets and 
squares) as single units and of the understanding of the topological relationships 
between these vista spaces, [5]. Decomposition of city space into a complete set 
of intersecting vista spaces produces a spatial network which we call the dual 
graph representation of a city. Therein, the relations between streets treated as 
nodes are traced through their junctions considered as edges. 

Dual city graphs are extensively investigated within the concept of space 
syntax, a theory developed in the late 1970s, that seeks to reveal the mutual 
effects of complex spatial urban networks on society and vice versa, [101 HI]- 
Spatial perception that shapes peoples understanding of how a place is organized 
determines eventually the pattern of local movement which is predicted by the 
space syntax method with surprising accuracy [12j . 



2.2 Euclidean space of undirected graphs associated to 
random walks 

Any graph representation naturally arises as the outcome of a categorization, 
when we abstract a real world system by eliminating all but one of its features 
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and by grouping together things (or places) sharing a common attribute. All 
elements called nodes that fall into one and the same group V are considered 
as essentially identical; permutations of them within the group are of no conse- 
quence. The symmetric group Sat consisting of all permutations of N elements 
{N being the cardinality of the set V) constitute the symmetry group of V. If 
we denote hy E C V x V the set of ordered pairs of nodes called edges, then 
a graph is a map G(V, E) : E K (we suppose that the graph has no 

multiple edges). 

The nodes of G{V, E) can be weighted with respect to some measure m = 
Xliey"^i'^i' specified by a set of positive numbers m.i > 0. The space i'^{m) 
of square-assumable functions with respect to the measure m is a Hilbert space 
TL. Among all linear operators defined on 7i, those invariant under the per- 
mutations of nodes are particularly interesting since they reflect the symmetry 
of the graph. Although there are infinitely many such operators, only those 
which maintain conservation of a quantity may describe a physical process. 
The Markov transition operators which share the property of probability con- 
servation considered in the theory of random walks on graphs are among them. 
Another example is given by the Laplace operators satisfying the mean value 
property {mass conservation) [T3] . 

Markov's operators on Hilbert space form the natural language of complex 
networks theory. Being defined on connected undirected graphs, a Markov tran- 
sition operator T has a unique equilibrium state tt ( stationary distribution of 
the random walk) such that ttT — tt and tt = limt^oo cr T* for any density a € Ti. 
(cj > 0, J2iev '^i = There is a unique measure TO,r = Tlii^v ""^'^^ related to 
the stationary distribution tt with respect to which the Markov operator T is 
self-adjoint, 

where is the adjoint operator. The orthonormal ordered set of real eigen- 
vectors '0i, I = 1 ... TV, of the symmetric operator T defines a basis in Ti. In the 
theory of random walks defined on graphs [151 HI] and in spectral graph theory 
[16j . the properties of graphs are studied in connection with the eigenvalues 
and eigenvectors of self-adjoint operators defined on them. In particular, the 
symmetric transition operator of the random walk defined on undirected graphs 
is Tij = 1 / ^Jki kj Hi ^ j. Its first eigenvector tpi belonging to the largest 
eigenvalue jii — 1, 

ipiT = V'l, i/'i,,; = TT^, (5) 
describes the local property of nodes (connectivity), tt^ = ki/2M, where 2M = 
X^iey while the remaining eigenvectors { tps }s=2 belonging to the eigenvalues 
1 > Ii2 > ■ ■ ■ fJ-N ^ describe the global connectedness of the graph. 

Markov's symmetric transition operator T defines a projection of any density 
a € Ti. on the eigenvector -ipi of the stationary distribution tt, 

aT = tpi + (7'^ T, CT^ = a — ipi, (6) 
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in which a-^ is the vector belonging to the orthogonal complement of ipi. Thus, 
it is clear that any two densities a, p Ti differ with respect to random walks 
only by their dynamical components, (cr — p)T^ = (cr^ — p^)T* for alH > 0. 
Therefore, we can define the distance || . . . ||t between any two densities which 
they acquire with respect to random walks by 

Ik-HIr - E {'^-p\T'\a-p). (7) 

i > 

or, using the spectral representation of T, 

II ||2 t, I ; \/ ; I \ - P\'^s){'^s\(7 - p) 

W^- pWt = l^l^^^s W - p\i^s){'>Ps\cf - p) = 2^ j— — 

t>0 s=2 s=2 

(8) 

where we have used Diracs bra-ket notations especially convenient in working 
with inner products and rank-one operators in Hilbert space. 
If we introduce in 'H{V) a new inner product by 

= 5: fi^^^iii^ (9) 

t>0 s=2 ^'^ 

for all CT, p e Ti{V), then ([8]) is nothing else but 



pIIt - \\<y\\l + \\p\\l-2{a,p)^, (10) 



where 



N 



l|2 



^ ( g I ) ( I g ) ^^^^ 



is the square of the norm of ct G W(^) with respect to random walks. We 
finish the description of the [N — l)-dimensional Euclidean space structure of G 
induced by random walks by mentioning that given two densities cr, p £ T-l{V), 
the angle between them can be introduced in the standard way, 

cosZ(p,a) = ^i^i^. (12) 

Random walks embed connected undirected graphs into Euclidean space. This 
embedding can be used in order to compare nodes and to retrace the optimal 
coarse-graining representations. Namely, let us consider the density 6i which 
equals 1 at the node i ^ V and zero for all other nodes. With respect to the 

— 1/2 

measure m^, it corresponds to the density Vi — tt^ 6i. Then, the square of 
the norm of Vi is given by 

imI-^Et^^ (13) 

s— 2 
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where 'ips,i is the -component of the eigenvector ips. In the theory of random 
walks [15] , the quantity p3)) expresses the access time to a target node quanti- 
fying the expected number of steps required for a random walker to reach the 
node i € V starting from an arbitrary node chosen randomly among all other 
nodes with respect to the stationary distribution tt. 

The Euclidean distance between any two nodes of the graph G induced by 
random walks, 

Klj ^ II - Vj ifj, , (14) 

is simply the commute times in theory of random walks and is equal to the 
expected number of steps required for a random walker starting at i G V" to 
visit j & V and then to return to i again, [T5] . 

It is important to mention that the cosine of an angle calculated in accor- 
dance to (|12p has the structure of Pearson's coefhcient of linear correlations that 
reveals it's natural statistical interpretation. Correlation properties of flows of 
random walkers passing by different paths have been remained beyond the scope 
of previous studies devoted to complex networks and random walks on graphs. 
The notion of angle between any two nodes in the graph arises naturally as soon 
as we become interested in the strength and direction of a linear relationship 
between two random variables, the flows of random walks moving through them. 
If the cosine of an angle (|12p is 1 (zero angles), there is an increasing linear rela- 
tionship between the flows of random walks through both nodes. Otherwise, if 
it is close to -1 (tt angle), there is a decreasing linear relationship. The correla- 
tion is {tt/2 angle) if the variables are linearly independent. It is important to 
mention that as usual the correlation between nodes does not necessary imply 
a direct causal relationship (an immediate connection) between them. 

2.3 Graph partitioning by random walks 

Visual segmentation of networks based on 3D representations of their dual 
graphs is not always feasible. The graph partitioning problem seeks to partition 
a weighted undirected graph G into n weakly connected components Fi, . . . r„ 
such that [J"^i r„ C G and cither their properties share some common trait 
or the graphs nodes belonging to them are close to each other according to 
some distance measure defined on nodes of the graph. A number of different 
graph partitioning strategies for undirected weighted graphs have been studied 
in connection with object recognition and learning in computer vision _17J. In 
statistics. Principal Component Analysis (PCA) is used for the reducing size 
of a data set. It is achieved by the optimal linear transformation retaining 
the subspace that has largest variance (a lower-order principal component) and 
ignoring higher-order ones [IHl [TH] . 

Given an operator S self-adjoint with respect to the measure m defined on a 
connected undirected graph G, it is well known that the eigenvectors of the sym- 
metric matrix S form an ordered orthonormal basis {(fik} with real eigenvalues 
Ml ^ • • ■ ^ I^N- The ordered orthogonal basis represents the directions of the 
variances of variables described by S. The number of components which may 
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be detected in a network with regard to a certain dynamical process defined on 
it depends upon the number of essential eigenvectors of S. 
Let us consider the normalized Laplace operator, [IB] . 

Lij — Sij — Tij, (15) 

where Tij is the symmetric Markov transition operator Tij. Its eigenvalues, 

= Ai < . . . < A„ < < . . . < Aat < 2, (16) 

can be interpreted as the inverse characteristic time scales of the diffusion pro- 
cess such that the smallest eigenvalues correspond to the stationary distribu- 
tion 77 together with the slowest diffusion modes involving the most significant 
amounts of flowing commodity. 

There is a simple time scale argument which we use in order to determine the 
number of applicable eigenvectors from the ordered orthogonal basis of eigen- 
vectors, [fi,...fAr]. It is obvious that while observing the network close to 
an equilibrium state during short time, we detect flows resulting from a large 
number of transient processes evolving toward the stationary distribution and 
being characterized by the relaxation times oc A^^. While measuring the flows 
in sufficiently long time r, we may discover just n different eigenmodes ()16p . 
In general, the longer is the time of measurements r, the less is the number 
of eigenvectors we have to take into account in network component analysis of 
the network. Should the time of measurements be fixed, we can determine the 
number n of required eigenvectors. 

In order to obtain the best quality segmentation, it is convenient to center 
the n primary eigenvectors. The centroid vector (representing the center of mass 
of the set [ fi, . . . f„ ]) is calculated as the arithmetic mean, 

n 

m = - E f^- (17) 

k=l 

Let us denote the n x N matrix of n centered eigenvectors by 

F = [fi — m, . . . fjv — m] . 

Then, the symmetric matrix of covariances between the entries of eigenvectors 
{ffc} is the product of F and its adjoint F^, 

F F^ 

Gov = (18) 

It is important to note that the correspondent Gram matrix F^ F /{N — 1) = 1 
due to the orthogonality of the basis eigenvectors. The main contributions in 
the symmetric matrix Gov are related to the groups of nodes 

Gov Mk = cTfeUfe, (19) 
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which can be identified by means of the eigenvectors { Ufc } associated to the first 
largest eigenvalues among ai > 02, ■ ■ ■ ,> <Jn- By ordering the eigenvectors 
in decreasing order (largest first) , we generate an ordered orthogonal basis with 
the first eigenvector having the direction of largest variance of the components 
of n eigenvectors {f/c}- Let us note that due to the structure of F only the 
first n — 1 eigenvalues (Jk are not trivial. In accordance to the standard PCA 
notation, the eigenvectors of the covariance matrix Ufc are called the principal 
directions of the network with respect to the diffusion process defined by the 
operator S. A low dimensional representation of the network is given by its 
principal directions [ui, . . . , u„_i ] , for n < N. 

Diagonal elements of the matrix Cov quantify the component variances of 
the eigenvectors [ fi , . . . f„ ] around their mean values p7|) and may be large 
essentially for large networks. Therefore, it is practical for us to use the stan- 
dardized correlation matrix, 

Corr,, = ^"^1 , (20) 



ov 



instead of the covariance matrix Cov. We emphasize that the diagonal elements 
of ((20)) equal 1, while the off-diagonal elements are the Pearson's coefficients of 
linear correlations, [20] . 

Let U be the orthonormal matrix which contains the eigenvectors {ufe}, 
k = l,...,n— 1, of the covariance (or correlation) matrix as the row vectors. 
These vectors form the orthogonal basis of the (n — l)-dimensional vector space, 
in which every variance (f^ — m) is represented by a point G 



gfe=U(ffc-m). (21) 

Then each original eigenvector ffe can be obtained from e M^"^^) by the 
inverse transformation, 

ffc = gfc + m. (22) 

Using transformations (pij) and we obtain the {n — l)-dimensional repre- 
sentation {(fk} ili^' of the A'^-dimensional basis vectors { f s } f^j^ in the form 

(Pfc = U^Uffc + (1-U^U) m, (23) 

that minimizes the mean-square error between G and fk & M^"^^^ for 
given n. 

Variances of eigenvectors { f fc } are positively correlated within a principal 
component of the transport network. Thus, the transition matrix U^U can be 
interpreted as the connectivity patterns acquired by the network with respect 
to the diffusion process. Two nodes, i and j, belong to one and the same 
principal component of the network if (UU^ ) . . > 0. By applying the Heaviside 
function, which is zero for negative argument and one for positive argument, 
to the elements of the transition matrix UU^, we derive the coarse-grained 
connectivity matrix of network components. 
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3 Thermodynamics of Transport Networks 



In the present section, we study the spectrum (|16p of the normalized Laplace 
operator (|15|) defined on a connected, undirected graph G by means of spectral 
methods strongly inspired by statistical mechanics. The obvious advantage of 
statistical mechanics is that statistical moments of random walks would acquire 
a " physical interpretation" in the framework of this thermodynamic formalism. 
The full description of transport networks in short time and small scales requires 
a high dimensional space, e.g. the knowledge of locations and velocities of all 
agents participating in transport processes. Being defined on a graph, the time 
evolution of such a system can be described by just a few dynamically relevant 
variables called reaction coordinates [H]. Identification of slow variables and 
dynamically meaningful reaction coordinates that capture the long time evolu- 
tion of transport systems is among the most important problems of transport 
network analysis. The spectrum of Laplace operator could have gaps indicating 
the time scales separation, that is, there are only a few "slow" time scales at 
which the transport network is meta-stable, with many "fast" modes describing 
the transient processes toward the slow modes. The methods of spectral graph 
theory allow detecting and separating "slow" and "fast" time scales giving rise 
to the component analysis of networks in which the primary eigenvalues play 
the essential role. 

3.1 Spectral function of Laplace operator 

Since the spectra of self-adjoint Laplace operators defined on the undirected 
graphs are non-negative they can be investigated by means of the characteris- 
tic functions which discriminate contributions from the largest eigenvalues of 
Laplace operator in favor of those from the minimal ones. These characteristic 
functions are usually associated with kernels. 

Given a self-adjoint operator L defined on a finite dimensional Hilbcrt space 
Ti the latter has an orthonormal basis {(t>k]k=i ^'^'i every element of the Hilbert 
space can be written in a unique way as a sum of multiples of these basis 
elements. The use of Borel's functional calculus for a positive semi-definite 
operator function f{L) allows us to define a Hilbert space Ti on via the dot 
product, 

{4>,4>)n^{<t>,f{L)4>), (24) 
with the kernel f~^{L). The kernels associated with the Laplace operator de- 
fined on an undirected graph G play the role of the Green functions describing 
long-range interactions between different diffusion modes induced by the graph 
structure (the main graph components). The Green function recently introduced 
in is specified by the parameter /3 > 0, 

(K^)^^. = exp(-/3L,,,), (25) 

and solve the diffusion equation defined on a connected, undirected graph G: 

roc rOQ 

g= Kpdl3= exp(-/3L) d/3 = L-\ (26) 
Jo Jo 



9 



with initial conditions being a source at node i £ V at P = [53] . The function 
(K^)^^. describes the expected number of particles that would accumulate at 
vertex j after a given amount of time if they have been injected at vertex i 
and diffused through the graph along the edges. The exponential diffusion 
kernel (|25p discriminate large eigenvalues that is preferable while performing 
estimation on the graph since it biases the estimate towards functions which 
vary little over the graph components [24]-[25j. 

The distribution of eigenvalues G [0, 2[ of the self-adjoint Laplace operator 
defined on the undirected graph G are characterized by the spectral density 
function, 

N 

p{X)^ySiX-X,). (27) 



fe=i 



However, it is more convenient to study the properties of non-negative spectra 
by means of the characteristic functions. 

The first characteristic function is given by the Laplace transform of the 
spectral density ([27|) . 

known as the canonical partition function, encoding the statistical properties of 
the system being in the thermodynamic equilibrium. The inverse temperature 
parameter /? > which can be considered either as an effective time scale in 
the problem (the number of spaces in the dual graph a walker passes through 
in one time step) or as the laziness parameter. 



3.2 Thermodynamics of urban networks 

Statistical mechanics provides a framework for relating the microscopic prop- 
erties of a system to its macroscopic or bulk properties. The statistical prop- 
erties of diffusions defined on undirected graphs can be described by means of 
canonical partition function (j28p . in which /3~^ plays the role of the inverse 
temperature. In statistical mechanics, the temperature arises from the fact that 
the system is in the thermal equilibrium with its environment that is achieved 
when two systems in thermal contact with each other cease to exchange energy 
by heat [26j . Equilibrium implies a state of balance and can be naturally in- 
terpreted as the stationary distribution tt in the framework of random walks 
defined on the undirected graphs. 

Bringing up the concept of the canonical ensemble it is possible to derive 
the probability Pi that random walkers will be in a certain "microstate" char- 
acterized by the spectral value A^: 

P. = Z-i(/3)exp(~/3A,). (29) 

Therefore, the partition function (|28p can be used to find the expected value of 
any microscopic properties of fiows of the random walkers, which can then be 
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related to some macroscopic variables related to the entire system. It is worth to 
mention that since random walkers neither have masses nor kinetic energy, the 
nontrivial macroscopic variables that can be derived from the partition function 
(|28p characterize the the certain structural properties of the graph with respect 
to the self-adjoint operator defined on it. 

We have discussed the thermodynamic potentials pertinent to space syntax 
in five different compact urban patterns in |27| . Two of them are situated on is- 
lands: Manhattan (with an almost regular grid-like city plan) and the network of 
Venice canals (Venice stretches across 122 small islands, and the canals serve the 
function of roads). We have also considered two organic cities founded shortly 
after the Crusades and developed within the medieval fortresses: Rothenburg ob 
der Tauber (the medieval Bavarian city preserving its original structure from the 
13*^ century) and the downtown of Bielefeld (Altstadt Bielefeld), an economic 
and cultural center of Eastern Westphalia. To supplement the study of urban 
canal networks, we have investigated that one in the city of Amsterdam. We do 
not reproduce the results reported in [27j in the present paper refereing inter- 
ested readers to |17] for details. Below, we just give an outline of the approach 
and study the thermodynamic parameters which have not been investigated in 
our previous work. 

The expected value of the "microscopic energy" (the averaged eigenvalue) 
held at constant temperature 

can be interpreted as the microscopic definition of the thermodynamic variable 
corresponding to the internal energy in statistical mechanics. Due to the com- 
plicated topology of streets and canals, the flows of random walkers exhibit 
spectral properties similar to that of a thermodynamic system characterized by 
a nontrivial internal energy (see Fig. [J). In Fig. [l] one can clearly see the dif- 
ference between the street patterns of the organic cities (Bielefeld, Rothenburg, 
canals of Amsterdam and Venice) and the street grid of Manhattan. Taking the 
derivative of (A) with respect to the parameter /3 in we have 

where D^{\) is the variance^ the measure of its statistical dispersion, indicating 
how the eigenvalues of the normalized Laplace operator are spread around the 
expected value U indicating the variability of the eigenvalues. The standard 
form of the usual deviations from the mean is the standard deviation, 



a = /d^. (32) 

A large standard deviation indicates that the data points are far from the mean 
and a small standard deviation indicates that they are clustered closely around 
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Figure 1: The expected values of the "microscopic energy" (the averaged eigenvalues) 
are calculated for the spectra of compact urban patterns. The slopes of internal 
energy curve for the highly regular street grid of Manhattan is steeper than those for 
the street patterns of the organic cities (Bielefeld, Rothenburg, canals of Amsterdam 
and Venice). 



the mean (see Fig. [2]). In thermodynamics, heat capacity is the rate of change 
of temperature as heat is added to a body at the given conditions and state of 
the body (foremost its temperature). Similarly to pressure, it is an extensive 
quantity being sensitive to the network size. Dividing heat capacity by the 
network size yields a specific heat capacity, which is an intensive quantity that 
is no longer dependent on N, and is now dependent on the graph structure, 

C^im (33) 



N \dT^ 

We have presented the specific heat capacity, as a function of temperature in 
the model of random walks defined on the different compact cities in Fig. [3] 

Note that the specific heat capacity is zero at zero temperature (/3 ~> oo), 
and rises to a maximal value as temperature grows up. It is interesting to 
note that the specific heat capacity in the random walks models defined on 
networks is dramatically different from that of most physical systems (like the 
multi-atomic gases or a crystalline solid phase); it is not a monotonous increas- 
ing function approaching the Dulong-Petit limit at higher temperature (the 
theoretical maximum of heat capacity) . The decay of specific heat at high tem- 
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Figure 2: The deviations of eigenvalues around the "microscopic energy" (the aver- 
aged eigenvalues) are calculated for the spectra of compact urban patterns. 



peratures {(3 0) results from the "freezing out" of some degrees of freedom 
as /3 — > (when random walkers rest at nodes most of the time). 

The maximal values of the specific heat capacity is an individual character- 
istics of a city structure. They are higher for the organic cities and lower for 
those planned in a regular grid. 

4 Directed and Interacting Transport Networks 

Many transport networks correspond to directed graphs. Power grids trans- 
porting electrical currents and driving directions assigned to city streets in order 
to optimize traffic in a city are the examples [28j . Traffic within large cities is 
formally organized with marked driving directions creating one-way streets. On 
those streets all traffic must flow in only one direction, but pedestrians on the 
sidewalks are generally not limited to one-way movement. It is well known that 
the use of one-way streets would greatly improve traffic flow since the speed of 
traffic is increased and intersections are simplified. 

Moreover, even if the network of city itineraries constitutes an undirected 
graph, there is a general principle that establishes who has the right to go 
first while crossing the road intersections or other conflicting parts of the road 
and who has to wait until the other driver does so. While assigning weights 
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Specific heat capacity of orlian textures 




Figure 3: The specific heat capacity, as a function of temperature in the model of 
random walks defined on the different urban textures. 



estimating traffic loading of itineraries on such a city graph, we could shortly 
discover that each road side should be characterized by its own weight depending 
upon the number of right-hand junctions the road has with other itineraries that 
is a directed weighted graph. 

The spectral approach for directed graphs has not been as well developed 
as for undirected graph. Indeed, it is rather difficult if ever possible to define 
a unique self-adjoint operator on directed graphs. In general, any node z in a 
directed graph G can have different number of in- neighbors and out-neighbors, 

kin{i) ^ fcout(*)- (34) 

In particular, a node i is a source if fcin(j) — 0, A:out(*) 7^ 0, and is a sink if 
kout{i) — 0, kin{i) 7^ 0. If the graph has neither sources nor sinks, it is called 
strongly connected. 

Despite asymmetric interactions represented by directed graphs are ubiqui- 
tous in many technological networks, they have not been yet very well studied 
in complex network theory. In general, the local structure of directed graphs is 
fundamentally different from that of undirected graphs. In particular, the diam- 
eters of directed networks can essentially exceeds that the one corresponding to 
the same networks regarded as undirected. A recent investigation into the local 
structure of directed networks [29j shows that directed networks often have very 
few short loops as compared to random models usual in contemporary theory 
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of complex systems. In undirected networks, the high density of short loops 
(high clustering coefficient) together with small graph diameter gives rise to the 
small- world effect |3D]. In directed networks, the correlation between number 
of incoming and outgoing edges modulates the expected number of short loops. 
In particular, it has been demonstrated in that if the values kin{i) and 
fcout(«) are not correlated, then the number of short loops is strongly reduced as 
compared to the case when both degrees are positively correlated. 



4.1 Random walks on directed graphs 

Finite random walks are defined on a strongly connected directed graph 
G{V,E) as finite vertex sequences tn — {vq, . . . (time forward) and ro' = 
{v-n, . • . , Vq} (time backward) such that each pair (w^-i, Vi) of vertices adjacent 
either in to or in ro' constitutes a directed edge Vi-i Vi in G. 

A time forward random walk is defined by the transition probability matrix 
[31] Pij for each pair of nodes i, j G G by 

0, otherwise, ^ ' 
which satisfies the probability conservation property: 

The definition (P5|) can be naturally extended for weighted graphs [3T] with 
Wij > 0, 

P. = (37) 

Matrices (|35l) and ([57)1 are real, but not symmetric and therefore have complex 
conjugated pairs of eigenvalues. For each pair of nodes i,j £ G, the forward 
transition probability is given by p'*' — (P*)ij that is equal zero, if G does not 
contain a directed path from i to j. 

Backwards time random walks are defined on the strongly connected directed 
graph G by the stochastic transition matrix 

p* _ J l/^in(i)5 j (38) 
'J 1^ 0, otherwise, ^ ' 

satisfying another probability conservation property 

E ^ = 1- (39) 

1, i^j 

It describes random walks unfolding backwards in time: should a random walker 
arrives at t = at a node Vq, then 

- (40) 
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defines the probability that t steps before it had originated from a node j. The 
matrix element (|40[) is zero, provided there is no directed path from j to i in G. 

If G is strongly connected and aperiodic, the random walk converges [321 
inn [Ml [M] to the on^?/ stationary distribution tt given by the Perron vector, 
ttP = TT. If the graph G is periodic, then the transition probability matrix P 
can have more than one eigenvalue with absolute value 1, |16j . The components 
of Perron's vector tt can be normalized in such a way that ^ ^- "^^"^ 

Perron vector for random walks defined on a strongly connected directed graph 
can have coordinates with exponentially small values, [31| . 

Given an aperiodic strongly connected graph G, a self-adjoint Laplace oper- 
ator L = can be defined [31] as 

where tt is a diagonal matrix with entries tt^. The matrix (|4ip is symmetric and 
has real non-negative eigenvalues = Ai < A2 < . . . < A^v and real eigenvectors. 

It can be easily proven ^35J that the Laplace operator (|4T|) defined on the 
aperiodic strongly connected graph G is equivalent to the Laplace operator 
defined on a symmetric undirected weighted graph G^j on the same vertex set 
with weights defined by 

Wij = TTiPij + TTjPji. (42) 

4.2 Bi-orthogonal decomposition of random walks defined 
on strongly connected directed graphs 

In order to define the self-adjoint Laplace operator (|4T|) on aperiodic strongly 
connected directed graphs, we have to know the stationary distributions of ran- 
dom walkers tt. Even if tt exists for a given directed graph G, it can be evaluated 
usually only numerically in polynomial time [32] . Stationary distributions on 
aperiodic general directed graphs are not so easy to describe since they are typ- 
ically non-local in sense that each coordinate tt^ would depend upon the entire 
subgraph (the number of spanning arborescences of G rooted at i [32]), but 
not on the local connectivity property of a node itself like it was in undirected 
graphs. Furthermore, if the greatest common divisor of its cycle lengths in G 
exceeds 1, then the transition probability matrices ([55)) and ([55]) can have sev- 
eral eigenvectors belonging to the largest eigenvalue 1, so that the definition 
([4T|) of Laplace operator seems to be questionable. 

Given a strongly connected directed graph G specified by the adjacency 
matrix ^ "^5' consider two random walks operators. A first transition 
operator represented by the matrix 

P = D„-\Ag, (43) 

in which Dout is a diagonal matrix with entries fcout (?) , describes the time for- 
ward random walks of the nearest neighbor type defined on G. Given a time 



16 



forward vertex sequence ro rooted at i £ G, the matrix element gives the 
probability that j € G is the vertex next to i in ro. A second transition operator, 
P*, is dynamically conjugated to (|^ . 

p* n^A^ 

where Di„ is a diagonal matrix with entries fcin(?). The transition operator (|44p 
describes random walks over time backward vertex sequences tr'. 

It is worth to mention that on undirected graphs P* = P, since fcin(i) = 
kout{i) for \/i € G and Aq — A J. While on directed graphs, P* is related to P 
by the transformation 

P = D„-\(PYDi„, (45) 

so that these operators are not adjoint, in general P^ 7^ P*. 
We can define two different measures 

= XI dugout M- = XI dcginOXi) (46) 

3 j 

associated with the out- and m-degrees of nodes of the directed graph. In 
accordance to ([^S]) , we define two Hilbert spaces 7i+ and Ti^ corresponding to 
the spaces of square summable functions, £^ and P by setting the 

norms as 

M\n± = 

where {■r)n± denotes the inner products with respect to measures (QB]). Then a 
function /(j) defined on the set of graph vertices is f-H_ (j) G H- if transformed 

by 

J-/(j)^M-7'/V(j) (47) 
and fu^ (j) G Ti.+ while being transformed accordingly to 

fn^ (j) - J+fij) ^ f^+j'^'fU)- (48) 

The obvious advantage of the measures against the natural counting mea- 
sure /io — J2iev tha.t the matrices of the transition operators P and P* are 
transformed under the change of measures as 



and become adjoint. 



P,, = J+'PJ-, P;^JI'P*J+, (49) 



(P*) EE (P^) - . , 

It is also important to note that 

P^-.U-^ H+ and P*:n+^ H- 
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We obtain the singular value dyadic expansion (biorthogonal decomposition in- 
troduced in [36l[37]) for the transition operator: 

N N 

^^As(psitpsi = ^^s\'Ps){tps\, (51) 
s=l s=l 

where < Ai < . . . Ajv and the functions (pk G and G 7i- are related by 
the Karhunen-Loeve dispersion [381 139] , 

P^(/?s = Asips (52) 
satisfying the orthogonality condition: 

{(Ps,fs')n+ = {i^s',i's)n- 5s' s- (53) 

Since the operators and Pj act between different Hilbert spaces, it is in- 
sufficient to solve just one equation in order to determine their eigenvectors ips 
and tps, [iQ]. Instead, two equations have to be solved, 

or, equivalently. 

The latter equation allows for a graph-theoretical interpretation. The block 
anti-diagonal operator matrix in the left hand side of (j55D describes random 
walks defined on a bipartite graph. Bipartite graphs contain two disjoint sets 
of vertices such that no edge has both end-points in the same set. However, in 
(l55|) . both sets are formed by one and the same nodes of the original graph G 
on which two different random walk processes specified by the operators and 
Pj are defined. 



It is obvious that any solution of the equation ([55]) is also a solution of the 
system 

Uip = AV, = AV, (56) 

in which U = PjP/^ and V = P^Pj, although the converse is not necessarily 
true. The self-adjoint nonnegative operators U : H- H- and V : H+ H+ 
share one and the same set of eigenvalues A^ £ [0,1], and the orthonormal 
functions {ifik} and {tpk} constitute the orthonormal basis for the Hilbert spaces 
Ti.+ and Ti- respectively. The Hilbert-Schmidt norm of both operators, 

N 

tr(p;P,)=tr(P,P;)=^A2 (57) 

s=l 

is a global characteristic of the directed graph. 

Provided the random walks are defined on a strongly connected directed 
graph G, let us consider the functions p'*''(s) G [0,1] x 2+ representing the 
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probability for finding a random walker at the node s, at time t. A random 
walker located at the source node s can reach the node s' through either nodes. 
Being transformed in accordance to (|47p and (I48p . these function takes the 
following forms: {p^P)h- = fJ--^^"^ P^'*\s) and [p^p)-^^ — fi'^^^^ p^*'> (s). Then, 
the self-adjoint operators U : 7Y_ H- and V ; 0.+ 7Y+ with the matrix 
elements 



Vfcout(s')fci„(s) ^fe„„^(j)fci„(j) 



(58) 



define the dynamical system 



J:seG{p'^^)^_Uss' = {p 



(59) 



4.3 Spectral analysis of self-adjoint operators defined on 
directed graphs 

The spectral properties of the self-adjoint operators U and V driving two 
Markov processes on strongly connected directed graphs and sharing the same 
non-negative eigenvalues g [0, 1] can be analyzed by the method of char- 
acteristic functions. Being defined on a strongly connected directed graph the 
spectral properties of self-adjoint operators U and V can be used in order to 
investigate the structure of the graph precisely as it is done in spectral graph 
theory for undirected graphs. 

The main goal of morphological analysis being applied to directed graphs is 
to detect the groups of nodes strongly correlated with respect to the random 
traffic that arrives at and departs from them. It is worth to mention that a 
node in a directed graph can have dramatically different numbers of incom- 
ing and outgoing links. As a consequence, nodes which could serve as a good 
source of random traffic for many other nodes in the graph, at the same time, 
could have an exponentially small probability to host a random walker. The 
self-adjoint operators U and V describe correlations between flows of random 
walkers entering and leaving nodes in a directed graph. In the framework of 
spectral approach, they are labelled by the eigenvalues A^, s = I,. . .N, and 
those correlations essential for coherence of different segments of the transport 
network correspond to the largest eigenvalues. The general method of PCA (see 
Sec. 12. 3p can be applied independently to U and V in order to detect segments 
of the graph coherent with respect to 7i + and 7i_ . 

The approach which we propose below for the analysis of coherent structures 
that arise in strongly connected directed graphs is similar to that one used in 
purpose of the spatiotemporal analysis of complex signals in [351 HO] and refers 
to the Karhuen-Loeve decomposition in classical signal analysis [38]. In the 
framework of bi-orthogonal decomposition discussed in [36j , the complex spatio- 
temporal signal has been decomposed into orthogonal temporal modes called 
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chronos and orthogonal spatial modes called topos. Then the spectral analysis 
of the phase-space of the dynamics and the spatio-temporal intermittency in par- 
ticular has naturally led to the notions of " energies" and " entropies" (temporal, 
spatial, and global) of signals. Each spatial mode has been associated with an 
instantaneous coherent structure which has a temporal evolution directly given 
by its corresponding temporal mode. In view of that the thermodynamic-like 
quantities had been used in order to describe the complicated spatio-temporal 
behavior of complex systems. 

In the present subsection, we demonstrate that a somewhat similar approach 
can also be applied to the spectral analysis of directed graphs. Namely, we will 
show that the morphological structure of directed graphs can be related to quan- 
tities extracted from bi-orthogonal decomposition of random walks defined on 
them. Furthermore, in this context, the temporal modes and spatial modes 
introduced in [36] are the eigenfunctions of correlation operators U and V. Al- 
though our approach can be viewed as a version of the signal analysis, it is 
fundamentally different from that in principle. By definition transition proba- 
bility operators satisfy the probability conservation property but it is in general 
not the case for the spatio-temporal signals generated by complex systems. 

All coherent segments of a directed graph participate independently in the 
Hilbert-Schmidt norm ([Ff|) of the self-adjoint operators U and V, 

N 

e(G) = ^A2. (60) 



Borrowing the terminology from theory of signals and [36j , we can call (j60p en- 
ergy, the only additive characteristic of the directed graph G. While introducing 
the projection operators (in Dirac's notation) by 

Pi+) = |V'.)(^.|, Pi-) = |^.)(^.|, s^l,...N, (61) 

we can decompose 2;(G) into two components related to the Hilbert spaces 7i+ 
and TC- : 

e(G) = (£(+)(G) + e:(-)(G) 

Using (j60p as normalization, we can consider the relative energy for each coher- 
ent structure of the directed graph by 

A2 

Cs - (63) 
e;(G) ^ ^ 

and, in the spirit of [35] , define the global entropy of coherent structures in the 
graph G as 



1 ^ 

^(G) = -j^5]e.lne„ s = l,...N, (64) 



s=l 
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which is independent on the graph size A'' due to the presence of the normalizing 
factor 1/lnA^, and can therefore be used in order to compare different directed 
graphs. The global entropy of the graph G is zero if all its nodes belong to 
one and the same coherent structure (i.e., only one eigenvalue 7^ 0). In the 
opposite case, Sj{G) ^ 1 if most of eigenvalues are degenerate. 

The relative energy (|63|l can also be decomposed into the H+- and Ti.-- 
components: 

A2m(±) 

A^ 

(£(±)(G) ^ ^ 

and then the partial entropies can be defined as 

i3W(G) = -i^EeWlne(±).. (66) 

The conclusion is that any strongly connected directed graph G can be con- 
sidered as a bipartite graph with respect to the in- and out-connectivity of 
nodes. The bi-orthogonal decomposition of random walks is then used in order 
to define the self-adjoint operators on directed graphs describing correlations 
between flows of random walkers which arrive at and leave the nodes. These 
self-adjoint operators share the non-negative real spectrum of eigenvalues, but 
different orthonormal sets of eigenvectors. The standard principal component 
analysis can be applied also to directed graphs. The global characteristics of the 
directed graph and its components can be obtained from the spectral properties 
of the self-adjoint operators. 



4.4 Self-adjoint operators for interacting networks 

The bi-orthogonal decomposition can also be implemented in order to de- 
termine coherent segments of two or more interacting networks defined on one 
and the same set of nodes y, |y| = N. 

Given two different strongly connected weighted directed graphs Gi and 
G2 specified on the same set of N vertices by the non-symmetric adjacency 
matrices A'^^\ A^^^ which entries are the edge weights, w^^'"^^ > 0, then the 
four transition operators of random walks can be defined on both networks as 

PM^(d(^^)"'a("), (p("))'^=(D(t))~'AW, a = 1,2, (67) 
where Dout/in are the diagonal matrices with the following entries: 

&J) - E - E " = 1'2- (68) 

i.j — >i i^i—^j 

We can define 4 different measures, 



(69) 
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and four Hilbert spaces Ti.^'^ associated with the spaces of square summable 

functions, £^ (/^i*^) , a = 1, 2. 

Then the transitions operators Pj"^ : ^ h'l^ and (pj"^)"^ : 7^^"' 

Ti^-* adjoint with respect to the measures /i^-* are defined by the following 
matrices: 



The spectral analysis of the above operators requires that four equations be 
solved: 

where a = 1,2 as usual. 




Figure 4: Self-adjoint operators for two interacting networks sharing the same set of 
nodes. 

Any solution {i^*-"-*, V"*-"'} of the system (|7T|) . up to the possible partial isome- 
trics, 

G("V'"^ = (72) 

also satisfies the system 



P«Tp(^)p(^'Tp(^V(^^ = (A(i)A(2))'V(^', 
P«p(^)Tp(2)p(2)T^(i) ^ (A(i)A(2))V(i), 

p(^)p(^)Tp(l)p(l)T^(2) ^ (A(1)A(2))2^(2), 



(73) 
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in which (A^^^A^^^) G [0, 1]. Operators in the l.h.s of the system (ffS]) describe 
correlations between flows of random walkers which go through vertices fol- 
lowing the links of either networks. They are represented by the non-negative 
symmetric matrices with orthonormal eigenvectors which can be subjected to 
the standard methods of the PCA analysis. Their spectrum can also be inves- 
tigated by the methods discussed in the previous subsection. 

It is convenient to represent the self-adjoint operators from the l.h.s. of 
(|73|) by the closed directed paths shown in the diagram in Fig. E) Being in the 
self-adjoint products of transition operators, Pjf^ corresponds to the flows of 
random walkers which depart from either networks, and P^"''^ is for those which 
arrive at the network a. From Fig. 0], it is clear that the self-adjoint operators 
in ([75]) represent all possible closed trajectories visiting both networks Ni and 

N2. 

In general, given a complex system consisting of rt > 1 interacting networks 
operating on the same set of nodes, we can define 2" self-adjoint operators 
related to the different modes of random walks. Then the set of network nodes 
can be separated into a number of essentially correlated segments with respect 
to each of self-adjoint operators. 

5 Discussion and Conclusion 

In the present paper, we have developed a self-consistent approach to com- 
plex transport networks based on the use of Markov chains defined on their 
graph representations. 

From Euler's time, urban design and townscape studies were the sources of 
inspiration for the network analysis and graph theory. It is common now that 
networks are the reality of urban renewals [H] . Flows of pedestrians and vehicles 
through a city are dependent on one another and that requests for organizing 
them in a network setting. 

The networking is structurally contagious. In order to be able to master 
a network effectively, an authority should also constitute a network structure, 
probably as complicated as the one that it supervises. The governance and 
maintenance units supporting urban network renewals, negotiating performance 
targets, taking decisions, financing and eventually implementing them also form 
networks. A complex network of city itineraries that we can experience in 
everyday life appears as the result of multiple complex interactions between a 
number of transport, social, and economical networks. 

It is intuitively clear that a complex network in equilibrium emerges from 
synergy and interplay between the topological structure shaped by a connected 
graph G with some positive measures (masses) appointed for the vertices and 
some positive weights assigned to the edges, the dynamics described by the set 
of operators defined on G, and the properties of the embedding physical space 
specified by the metric length distances of the edges. 

The approach we have discussed in the present paper helps to define an 
equilibrium state for complex transport networks and investigate its properties. 
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